The present invention relates to a system, a device, and vehicle for recording panoramic images.
A panoramic image is an image with an elongated field of view (FOV). Normally, an optical camera records an image with a limited field of view as defined by the photosensitive component and lens system of the camera. In order to obtain a image of a wider format, one has to use different additional lenses, such as a fisheye lens, or one has to combine multiple images to provide the impression of one large image. The latter process requires stitching of the images. Normally, the multiple images have a slight overlap in their field of view such that an interpolation process can be used to determine the image properties in the overlapping region.
A drawback of combining multiple images is that it is prone to parallax errors. These errors originate from the fact that the multiple images are not recorded from the same position. A solution to this problem is to use rotatable cameras which are mounted on a stand. By making sure that the entrance pupil of the camera remains substantially at the same position, albeit rotated, one can reduce parallax errors to an acceptable degree.
Using a single camera to record a plurality of images that are later combined into a single panoramic image limits the applicability of the technique to the recording of static environments only. Moreover, such technique is not suitable for recording movies as this would require the camera to rotate at impractical speeds. Moreover, such system is not suitable for applications in which the camera itself is moving. Examples of such applications are the recording of panoramic images for navigational systems or derivatives thereof such as Google Street View.
A solution to this problem has been disclosed in U.S. 2002/0089765 A1. The system described therein uses a reflective pyramid in which the reflective sidewalls are under a 45° angle with respect to the base of the pyramid. Cameras are positioned relative to each of the sidewalls such that each camera appears to record an image from a virtual reference point inside the pyramid. In other words, the images recorded by these cameras are identical to the images that would have been recorded by a virtual optical camera positioned at the reference point. Here, reference point corresponds to the position of the entrance pupil of the virtual camera. Using a pyramid with four sidewalls therefore results in four images from four different orientations which can later on be combined into one panoramic image. Moreover, the images making up the larger panoramic image, the so called partial images, appear to have been recorded at substantially the same position of the entrance pupil, thereby reducing or eliminating parallax errors.
A recent trend or desire is to obtain panoramic images with very high resolution, for instance for making detailed measurements in those images. Furthermore, to be able to derive useful information from these images, they need to be metrically correct. These desires can only be partially met by the abovementioned system. Given an optical camera with a predefined resolution one can increase the resolution of the panoramic image by increasing the number of cameras used. This allows a camera to use its full resolution for a relatively small field of view.
A solution to this problem is presented in EP 0 982 946 A1. This system resembles that of U.S. 2002/0089765 A1 with the exception that it describes how different pyramids may be stacked in the vertical direction.
A drawback of the abovementioned system is that it is difficult to stitch the various partial images that are capture by the cameras. To ensure proper stitching, an overlapping region in adjacent partial images is preferred. By using cameras with overlapping FOVs, as disclosed in EP 0 982 946 A1, such overlap can be realized. Unfortunately, the regions of overlap correspond to physical boundaries of the system, such as edges of mirrors or lenses. These boundaries introduce unwanted deformations in the partial images, thereby deteriorating the stitching process.